Deliberation and the Wisdom of Crowds

Deliberation increases the wisdom of crowds

The aggregation of many independent estimates can outperform the most accurate individual judgment. This centenarian finding , popularly known as the “wisdom of crowds”, has been applied to problems ranging from the diagnosis of cancer to financial forecasting. It is widely believed that social influence undermines collective wisdom by reducing the diversity of opinions within the crowd. Here, ...

Evaluating the Wisdom of Crowds

Estimating and forecasting are difficult tasks. This is true whether the activity requires the determination of uncertain future event outcomes, or whether the estimation effort is complex in itself and based on insufficient information. Consequently, such tasks are frequently assigned to experts. Surprisingly, recent research suggests that collectives of non-experts can outperform individual e...

The Multidimensional Wisdom of Crowds

Distributing labeling tasks among hundreds or thousands of annotators is an increasingly important method for annotating large datasets. We present a method for estimating the underlying value (e.g. the class) of each image from (noisy) annotations provided by multiple annotators. Our method is based on a model of the image formation and annotation process. Each image has different characterist...

Harnessing the Wisdom of Crowds*

We empirically examine the impact of herding on the accuracy of consensus earnings forecasts from a crowd-based platform (Estimize.com). We monitor user information set by tracking user viewing activities and running experiments. We find that the more public information a user views, the less weight she puts on her private information. While this improves the accuracy of individual forecasts, i...

The “Wisdom of Crowds” Effect